Custom Report
1 Intoduction
This report is one of 493,845 that I will make, and one of 104,070,413 that could be made.
I “toke” the 1.4 TB Linked-In data that was breached in 2020, and turned it into some insights to power my job HUNT.
The insights I could share in this report, that are also related to my goals, are:
- Industry base recruitment trend.
- Company base workforce timeline.
- Current/part workforce info:
- Basic info: Name, job title, status, social link. I could add geo-location for some that have the data, but it would look creepy.
- Their work period.
- Their experiences.
2 About me
Salutations; I’m Joseph, a self-taught data analyst, engineer, and scraper.
Despite life’s challenges, my goal remains a remote job, either full or part-time, and having friends to tackle the challenges of this changing world with.
To show my skills and dedication, I made this project that yielded this tailored report.
3 About the project
3.1 How this project comes to life?
You would know by now, from my email, that I am hunting for a job.
About a year ago, I scraped contact info from Google Map to get my first job. Later I scraped contact from Linked-In website… you can check how that went in here.
Recently, I finally got to learning SQL because of DuckDB, it is a software that allows you to process big data in your local machine by using storage space as RAM; Then I remembered about a leaked Linked-In data that I couldn’t process.
Thus my journey started to learn SQL, process the data, and make something out of it.
3.2 The process
The process was done in my local machine, and it was as followed.
3.2.1 Downloaded the leaked data
I downloaded the data from a torrent.
There was around 700 .gz file, each is around 280 Mb; 196 GB in total.
Each .gz file contain a 2 GB file; 1.4 TB in total.
Each file have multiple lines, and each one of them is a JSON; Not the file is a JSON, it just have multiple JSONs, one in each line.
3.2.2 Processing the weird data
I this phase I created a script that automatically open an archive, process the file, and save it as a Parquet file with compression level of 22.
I used Python, Pathlib, Polars, and a lot of patience.
The process toke around 20 minutes per file, in total it toke around three weeks (I had to shutdown my PC at night) The result was 700 parquet files, each is around 190 Mb; 133 GB in total.
3.2.3 Making relational database
The data in the datasets were nested, especially the “experience” field, it had the experience of a person and the company info; The problem is that the company info get repeated multiple tiles, across all datasets.
Making a relational database will solve this, and make the exploratory data analysis easier.
The code was split in two:
1. I used Polars to split each of the 700 datasets into mini relational databases.
2. I used DuckDB to merge all the mini relational databases and remove duplicates in some, mainly company and university information’s.
The result was a relational database that is 73 GB in size; From 1.4 TB to 73 GB.
All of this is using my PC, so servers were harmed, only my CPU fan and my ear.
3.2.4 Filter
I filtered out companies base on their industry, country, and whether I have the email of one of the higher ups.
4 General graphs
4.1 market research indestry’s yearly new recruit count
4.2 kadence international’s workforce status over the years
5 Workforce sample
5.1 Adhinda Hanifah
Job title: Qualitative research intern
Associated: False
Socials: https://linkedin.com/in/adhinda-milla-hanifah-5baaa218b
5.1.1 Adhinda Hanifah’s working period at kadence international
5.1.2 Gantt plot of Adhinda Hanifah’s experience
5.2 Aji Pitono
Job title: Dp supervisor
Associated: True
Socials: https://linkedin.com/in/aji-pitono-54035170
5.2.1 Aji Pitono’s working period at kadence international
5.2.2 Gantt plot of Aji Pitono’s experience
5.3 Alyssa Piper
Job title: Insight executive
Associated: True
Socials: https://facebook.com/alyssalynnpiper | https://linkedin.com/in/alyssalynnpiper
5.3.1 Alyssa Piper’s working period at kadence international
5.3.2 Gantt plot of Alyssa Piper’s experience
5.4 Amy Lo
Job title: Insight manager
Associated: True
Socials: https://linkedin.com/in/amy-lo-868a5641
5.4.1 Amy Lo’s working period at kadence international
5.4.2 Gantt plot of Amy Lo’s experience
5.5 Anand Mevada
Job title: Edc
Associated: True
Socials: https://linkedin.com/in/anand-mevada-92481837
5.5.1 Anand Mevada’s working period at kadence international
5.5.2 Gantt plot of Anand Mevada’s experience
5.6 Ankita Parmar
Job title: Kadence
Associated: True
Socials: https://linkedin.com/in/ankita-parmar-92728238
5.6.1 Ankita Parmar’s working period at kadence international
5.6.2 Gantt plot of Ankita Parmar’s experience
5.7 Anshuman Saha
Job title: Operations director
Associated: False
Socials: https://linkedin.com/in/anshumansaha | https://angel.co/anshuman-saha
5.7.1 Anshuman Saha’s working period at kadence international
5.7.2 Gantt plot of Anshuman Saha’s experience
5.8 Anthony Rowbottom
Job title: Research executive
Associated: False
Socials: https://twitter.com/arowbottom | https://facebook.com/anthony.rowbottom | https://linkedin.com/in/anthonyrowbottom | https://linkedin.com/in/anthony-rowbottom-4551bb10
5.8.1 Anthony Rowbottom’s working period at kadence international
5.8.2 Gantt plot of Anthony Rowbottom’s experience
5.9 Bhairava Sb
Job title: Operation excutive
Associated: True
Socials: https://linkedin.com/in/bhairava-sb-6475a2129
5.9.1 Bhairava Sb’s working period at kadence international
5.9.2 Gantt plot of Bhairava Sb’s experience
5.10 Dadang Stiawan
Job title: Associate research director
Associated: True
Socials: https://linkedin.com/in/dadang-stiawan-68b1a759
5.10.1 Dadang Stiawan’s working period at kadence international
5.10.2 Gantt plot of Dadang Stiawan’s experience
5.11 Dara Ningtyas
Job title: Market research intern
Associated: False
Socials: https://linkedin.com/in/dara-ayu-ningtyas
5.11.1 Dara Ningtyas’s working period at kadence international
5.11.2 Gantt plot of Dara Ningtyas’s experience
5.12 Destria Sitompul
Job title: Quantitative research executive
Associated: False
Socials: https://linkedin.com/in/destria-sitompul-a1156517
5.12.1 Destria Sitompul’s working period at kadence international
5.12.2 Gantt plot of Destria Sitompul’s experience
5.13 Diana Dobson
Job title: Human resources manager
Associated: False
Socials: https://linkedin.com/in/diana-r-dobson-58528273
5.13.1 Diana Dobson’s working period at kadence international
5.13.2 Gantt plot of Diana Dobson’s experience
5.14 Dipankara Surjadi
Job title: Senior research manager
Associated: True
Socials: https://linkedin.com/in/dipankara-surjadi-a3436238
5.14.1 Dipankara Surjadi’s working period at kadence international
5.14.2 Gantt plot of Dipankara Surjadi’s experience
5.15 Divesh Charan
Job title: Avp- asia fieldwork operations
Associated: True
Socials: https://linkedin.com/in/divesh-charan-93008214 | https://facebook.com/divesh.charan
5.15.1 Divesh Charan’s working period at kadence international
5.15.2 Gantt plot of Divesh Charan’s experience
5.16 Ebenezer Ebenezer.Kadence
Job title: Operations manager
Associated: True
Socials: https://linkedin.com/in/ebenezer-ebenezerkadence-75770848
5.16.1 Ebenezer Ebenezer.Kadence’s working period at kadence international
5.16.2 Gantt plot of Ebenezer Ebenezer.Kadence’s experience
5.17 Ellie Tehrani
Job title: Global director - data solutions
Associated: True
Socials: https://linkedin.com/in/ellie-tehrani-6437253b
5.17.1 Ellie Tehrani’s working period at kadence international
5.17.2 Gantt plot of Ellie Tehrani’s experience
5.18 Fahmi Afandi
Job title: Director
Associated: True
Socials: https://linkedin.com/in/fahmi-afandi-43659b8
5.18.1 Fahmi Afandi’s working period at kadence international
5.18.2 Gantt plot of Fahmi Afandi’s experience
5.19 Febri Nugraha
Job title: Managing director
Associated: True
Socials: https://linkedin.com/in/febri-nugraha-3a228743 | https://twitter.com/febri_nugraha
5.19.1 Febri Nugraha’s working period at kadence international
5.19.2 Gantt plot of Febri Nugraha’s experience
5.20 Ianne Lausin
Job title: Human resources officer
Associated: True
Socials: https://linkedin.com/in/iannelausin
5.20.1 Ianne Lausin’s working period at kadence international
5.20.2 Gantt plot of Ianne Lausin’s experience
5.21 Ibansara Lyngdoh
Job title: Kadence international pvt ltd
Associated: True
Socials: https://linkedin.com/in/ibansara-lyngdoh-a1400042
5.21.1 Ibansara Lyngdoh’s working period at kadence international
5.21.2 Gantt plot of Ibansara Lyngdoh’s experience
5.22 Iris Lorenzo
Job title: Managing director, kadence philippines
Associated: True
Socials: https://linkedin.com/in/ilorenzo
5.22.1 Iris Lorenzo’s working period at kadence international
5.22.2 Gantt plot of Iris Lorenzo’s experience
5.23 Jitendra Kumar
Job title: Telly caller
Associated: True
Socials: https://linkedin.com/in/jitendra-kumar-14a162104
5.23.1 Jitendra Kumar’s working period at kadence international
5.23.2 Gantt plot of Jitendra Kumar’s experience
5.24 Jonathan Lim
Job title: Data analysis manager
Associated: True
Socials: https://linkedin.com/in/jonathan-lim-3b77655a
5.24.1 Jonathan Lim’s working period at kadence international
5.24.2 Gantt plot of Jonathan Lim’s experience
5.25 Jorge Caicedo
Job title: ****
Associated: True
Socials: https://linkedin.com/in/jorge-caicedo-10565024
5.25.1 Jorge Caicedo’s working period at kadence international
5.25.2 Gantt plot of Jorge Caicedo’s experience
5.26 Karl Wagner
Job title: Data management director
Associated: True
Socials: https://linkedin.com/in/karl-wagner-91a73211 | https://twitter.com/kwagner73
5.26.1 Karl Wagner’s working period at kadence international
5.26.2 Gantt plot of Karl Wagner’s experience
5.27 Katrin Scheibert
Job title: Design and video manager
Associated: True
Socials: https://linkedin.com/in/katrin-scheibert-3a523323
5.27.1 Katrin Scheibert’s working period at kadence international
5.27.2 Gantt plot of Katrin Scheibert’s experience
5.28 Kit Wong
Job title: Senior insight manager
Associated: True
Socials: https://linkedin.com/in/kit-wong-69a2861b
5.28.1 Kit Wong’s working period at kadence international
5.28.2 Gantt plot of Kit Wong’s experience
5.29 Klaudia Scibior
Job title: Graduate insight executive
Associated: True
Socials: https://linkedin.com/in/klaudia-scibior-747a0274
5.29.1 Klaudia Scibior’s working period at kadence international
5.29.2 Gantt plot of Klaudia Scibior’s experience
5.30 Kunal Kapoor
Job title: Analyst
Associated: True
Socials: https://linkedin.com/in/kunal-kapoor-615b3018
5.30.1 Kunal Kapoor’s working period at kadence international
5.30.2 Gantt plot of Kunal Kapoor’s experience
5.31 Linda Trieu
Job title: Market research analyst
Associated: True
Socials: https://facebook.com/letstaysotrieu | https://linkedin.com/in/linda-trieu-9a699214a
5.31.1 Linda Trieu’s working period at kadence international
5.31.2 Gantt plot of Linda Trieu’s experience
5.32 Lizzy Pottinger
Job title: Associate insight director
Associated: True
Socials: https://linkedin.com/in/lizzy-pottinger-48679726
5.32.1 Lizzy Pottinger’s working period at kadence international
5.32.2 Gantt plot of Lizzy Pottinger’s experience
5.33 Louis Jimenez
Job title: Territory manager
Associated: True
Socials: https://linkedin.com/in/louis-jimenez-444a0446
5.33.1 Louis Jimenez’s working period at kadence international
5.33.2 Gantt plot of Louis Jimenez’s experience
5.34 Manpreet Singh
Job title: Edc
Associated: True
Socials: https://linkedin.com/in/manpreet-singh-9bb80944
5.34.1 Manpreet Singh’s working period at kadence international
5.34.2 Gantt plot of Manpreet Singh’s experience
5.35 Maria Seddaaiu
Job title: Market research executive
Associated: False
Socials: https://linkedin.com/in/maria-giovanna-seddaaiu-b0680112b
5.35.1 Maria Seddaaiu’s working period at kadence international
5.35.2 Gantt plot of Maria Seddaaiu’s experience
5.36 Mentari Rahmalah
Job title: Personal assistant to managing director
Associated: True
Socials: https://linkedin.com/in/mentari-meida-rahmalah-2aba3945
5.36.1 Mentari Rahmalah’s working period at kadence international
5.36.2 Gantt plot of Mentari Rahmalah’s experience
5.37 Michael Felix
Job title: Market research project coordinator
Associated: True
Socials: https://linkedin.com/in/michael-felix-0294b8a8
5.37.1 Michael Felix’s working period at kadence international
5.37.2 Gantt plot of Michael Felix’s experience
5.38 Mofatlal Yadav
Job title: Student
Associated: True
Socials: https://linkedin.com/in/mofatlal-yadav-ba748949
5.38.1 Mofatlal Yadav’s working period at kadence international
5.38.2 Gantt plot of Mofatlal Yadav’s experience
5.39 Motoka Mori
Job title: Business planning manager
Associated: True
Socials: https://linkedin.com/in/motoka-mori-73a6b620
5.39.1 Motoka Mori’s working period at kadence international
5.39.2 Gantt plot of Motoka Mori’s experience
5.40 Natalie Lewis-Marshon
Job title: Office manager and finance administrator
Associated: True
Socials: https://linkedin.com/in/natalie-lewis-marshon-12398830
5.40.1 Natalie Lewis-Marshon’s working period at kadence international
5.40.2 Gantt plot of Natalie Lewis-Marshon’s experience
5.41 Natalie Zatz
Job title: Insight manager
Associated: True
Socials: https://linkedin.com/in/natalie-zatz-95043016
5.41.1 Natalie Zatz’s working period at kadence international
5.41.2 Gantt plot of Natalie Zatz’s experience
5.42 Nguyen Trang
Job title: Country head
Associated: True
Socials: https://linkedin.com/in/ivy-trang-nguyen-010071b
5.42.1 Nguyen Trang’s working period at kadence international
5.42.2 Gantt plot of Nguyen Trang’s experience
5.43 Nikhil Raj
Job title: Insights manager
Associated: False
Socials: https://linkedin.com/in/nikhil-raj-7a228416 | https://twitter.com/nikhiltwitraj
5.43.1 Nikhil Raj’s working period at kadence international
5.43.2 Gantt plot of Nikhil Raj’s experience
5.44 Parth Amin
Job title: Data collector
Associated: False
Socials: https://linkedin.com/in/parth-amin-031248128
5.44.1 Parth Amin’s working period at kadence international
5.44.2 Gantt plot of Parth Amin’s experience
5.45 Parveen Bhatia
Job title: Project manager international business
Associated: False
Socials: https://linkedin.com/in/parveen-tara-bhatia-2a924015
5.45.1 Parveen Bhatia’s working period at kadence international
5.45.2 Gantt plot of Parveen Bhatia’s experience
5.46 Patrick Conlon
Job title: Motion graphic designer
Associated: True
Socials: https://linkedin.com/in/patrick-conlon-526bb12b
5.46.1 Patrick Conlon’s working period at kadence international
5.46.2 Gantt plot of Patrick Conlon’s experience
5.47 Ram Rohit
Job title: Supp
Associated: True
Socials: https://linkedin.com/in/ram-rohit-26551841
5.47.1 Ram Rohit’s working period at kadence international
5.47.2 Gantt plot of Ram Rohit’s experience
5.48 Rupert Sinclair
Job title: Insight director
Associated: True
Socials: https://linkedin.com/in/rupert-sinclair-66129122 | https://linkedin.com/in/rupert-sinclair-8424268
5.48.1 Rupert Sinclair’s working period at kadence international
5.48.2 Gantt plot of Rupert Sinclair’s experience
5.49 Santhya Rani
Job title: Quantitative researcher
Associated: True
Socials: https://linkedin.com/in/santhya-rani-a562b05b
5.49.1 Santhya Rani’s working period at kadence international
5.49.2 Gantt plot of Santhya Rani’s experience
5.50 Sarah Serbun
Job title: Insight director, us qualitative lead
Associated: True
Socials: https://linkedin.com/in/sarah-serbun-3382626 | https://facebook.com/sarah.serbun
5.50.1 Sarah Serbun’s working period at kadence international
5.50.2 Gantt plot of Sarah Serbun’s experience
5.51 Shaibal Ghosh
Job title: Associate director business development
Associated: True
Socials: https://linkedin.com/in/shaibal-ghosh-a2a29a131
5.51.1 Shaibal Ghosh’s working period at kadence international
5.51.2 Gantt plot of Shaibal Ghosh’s experience
5.52 Simona Farrell
Job title: Finance manager
Associated: True
Socials: https://linkedin.com/in/simona-borriero-farrell-1ba8511 | https://linkedin.com/in/simona-farrell-1ba8511
5.52.1 Simona Farrell’s working period at kadence international
5.52.2 Gantt plot of Simona Farrell’s experience
5.53 Sunil Jha
Job title: Edc
Associated: True
Socials: https://linkedin.com/in/sunil-jha-86214539
5.53.1 Sunil Jha’s working period at kadence international
5.53.2 Gantt plot of Sunil Jha’s experience
5.54 Sushil Ojha
Job title: Group account director
Associated: True
Socials: https://facebook.com/sushil.ojha.1980 | https://linkedin.com/in/sushil-ojha-61923ba
5.54.1 Sushil Ojha’s working period at kadence international
5.54.2 Gantt plot of Sushil Ojha’s experience
5.55 Teo Jacob
Job title: Operations director
Associated: True
Socials: https://linkedin.com/in/teo-jacob-60b52420
5.55.1 Teo Jacob’s working period at kadence international
5.55.2 Gantt plot of Teo Jacob’s experience
5.56 Trang Giang
Job title: Research manager
Associated: True
Socials: https://linkedin.com/in/michelle-giang-b34a4480
5.56.1 Trang Giang’s working period at kadence international
5.56.2 Gantt plot of Trang Giang’s experience
5.57 Tulika Sheel
Job title: Associate vice president - insights
Associated: True
Socials: https://linkedin.com/in/tulika-sheel-30b8405 | https://linkedin.com/in/tulikasheel | https://twitter.com/tulikasheel
5.57.1 Tulika Sheel’s working period at kadence international
5.57.2 Gantt plot of Tulika Sheel’s experience
5.58 Valintina Lim
Job title: Accounts executivecum office manager
Associated: True
Socials: https://linkedin.com/in/valintina-lim-5b848b86
5.58.1 Valintina Lim’s working period at kadence international
5.58.2 Gantt plot of Valintina Lim’s experience
5.59 Vivek Goel
Job title: Assistant vice president
Associated: True
Socials: https://linkedin.com/in/vivek-goel-98a75a4
5.59.1 Vivek Goel’s working period at kadence international
5.59.2 Gantt plot of Vivek Goel’s experience
5.60 Vivian Wong
Job title: Associate director
Associated: True
Socials: https://linkedin.com/in/vivian-wong-7ab10337